A New Service for Increasing the Effectiveness of Network Address Blacklists

We recently established a new experimental Internet service for creating customized source address blacklists for DShield.org contributors. This new service utilizes a radically different approach to blacklist formulation that we refer to as Highly Predictive Blacklists (HPB). A highly predictive blacklist is a list of malicious Internet addresses that is formulated through an analysis of the roughly 30 million firewall log entries that are contributed to the DShield repository each day from across the Internet. The HPB service employs a link analysis algorithm similar to the Google PageRank scheme to crosscompare contributors in search of overlaps among the attackers they report. The attacker addresses included within an HPB are selected by favoring the inclusion of those addresses that have been encountered by contributors who share degrees of overlap with the target HPB owner. Our experiments show that highly predictive blacklist entries consistently yield filters that are exercised at higher rates than those from conventional blacklist methods. In addition, this increase in blacklist filter “hit rates” can last multiple days into the future. In this paper, we provide an overview of our algorithm and present our usage experiences. We discuss the envisioned benefits that we believe HPBs can provide toward reducing unwanted communications for those networks that utilize this service.